Non Uniform Selection of Solutions for Upper Bounding the 3-SAT 

Threshold* 



Yacine Boufkhad and Thomas Hugel 
boufkhadOliaf a. jussieu. f r 
thomas . hugelOliaf a . j ussieu . f r 

LIAFA - Universite Denis Diderot Paris 7 - CNRS** 
Case 7014 
F- 75205 Paris Cedex 13 

^ ' Abstract. We give a new insight into the upper bounding of the 3-SAT threshold by the first moment method. 

' ^ , The best criteria developed so far to select the solutions to be counted discriminate among neighboring solutions 

■ on the basis of uniform information about each individual free variable. What we mean by uniform information, 
^T^l is information which does not depend on the solution: e.g. the number of positive/negative occurrences of the 

■^Uj , considered variable. What is new in our approach is that we use non uniform information about variables. 

\^ ' Thus we are able to make a more precise tuning, resulting in a slight improvement on upper bounding the 

3-SAT threshold for various models of formulas defined by their distributions. 

1 Introduction 

We consider the phase transition phenomenon that occurs in some random satisfiability problems, where the prob- 
^ ■ ability of satisfiability for a random formula suddenly goes from 1 to at a given ratio It was first 

experimentally observed that this transition would occur at a ratio near 4.25 for the standard 3-SAT model (see 
', |l])- The same kind of transition was also observed in some variants of the standard model, e.g. when occurrences 

■ and signs of variables are balanced (see [2J). 

I The first important step towards the quest of the threshold is the work of Friedgut and Bourgain [3] establishing 

. that the width of the transition window tends to zero as the number of variables tends to infinity. 

An important breakthrough was then made by Achlioptas and Peres [4J : using a sophisticated technique based on 
the second moment method they located asymptotically the threshold of k-SAT for large constant k at 2*'' In 2 — O (fc). 
' However in the particular case of 3-SAT, there remains a gap between established lower and upper bounds. 

The cornerstone method used for 25 years in order to establish upper bounds of the 3-SAT threshold is the so 
called first moment method. Indeed we are interested in the probability that a formula has some solutions, but that 
probability is currently out of reach of human-tractable calculations; however the moments under this probability 
are much easier to estimate. The first moment method consists in bounding the probability we are interested in 
by the first moment of a certain quantity X under this probability. The simplest quantity X one can imagine as a 
5h ' candidate for the first moment method is the number of solutions. This gives an upper bound of 5.191 [5J, which is 
far above the experimentally observed threshold at around 4.25. There has been ever since lots of efforts [6 7 8 9 lOj 
intended to lower this upper bound by removing as many solutions as possible from the counted quantity X, the 
only requirement of the first moment method being to count at least 1 solution whenever a formula is satisfiable; 
thus the technique is to count only particular solutions, designed to be present whenever there is a solution, and 
not too complicated to count. 

We obtain some new upper bounds in a variety of models of 3-CNF formulas (which we introduce later in section 
12. ip . In the particular case of the standard model we get an upper bound of 4.500. We must mention here the work 
of Diaz et al. |llj ; gathering the technique of |10|12| with a pure literal elimination and a filtering on the typicality 
of clauses, they got an upper bound of 4.490. The fact is that our new technique is quite compatible with the pure 
literal elimination and the filtering on the typicality of clauses, but we only aim at emphasizing the positive effect 
of our new technique for selecting solutions, by comparing it to previous analogous techniques in several models of 
formulas. 

The best implementations of the first moment method approximating the threshold of 3-SAT use local relation- 
ships between solutions, which involves solutions agreeing on the values of all variables but a constant number of 
them, in general one variable [5] or two [5]. 
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We shall consider the set of solutions with local relationship as a graph which nodes are the solutions and an 
edge exists between two solutions if and only if both solutions agree on the values of all variables except one. Each 
edge will be labelled by the variable differing between both solutions. 

For example the formula 

^={aV6Vc,aVcVd,aVcVfi,aV6Vd,6VcVd,aV6V(i,aV6Vc} 

has 7 solutions that can be represented by the non oriented graph of figure [1] 

The techniques used so far amount to making an acyclic orientation of the 
above graph and to counting only the minimal solutions (those that do not have 
outgoing edges). The least is the number of minimal solutions the best is the 
upper bound obtained. In general, any graph can be oriented so as to obtain 
only one minimal element for every connected component (e.g. by a depth first 
search), but this orientation is obtained thanks to a sophisticated algorithm that 
is aware of the whole graph while in our case, the orientation must be decided 
locally. 

The very first orientation [8'9] consisted in orienting an edge from the solution 
where the label variable is assigned to the one where it is 1 regardless of which 
variable is considered. Later, in |12|11| . an edge is oriented towards the value 
that makes true the most literals and this can be known thanks to the syntactic 
property of the number of occurrences of each variable in the formula. In both 
these types of orientation, the edges having the same labels are oriented the 
same way (e.g. from to 1) anywhere in the graph. So we call such orientations 




uniform (see Figure 2(a)) 



Fig. 1. Graph of solutions for 
formula F. The label of an edge 
is the name of the variable differ- 
ing between both solutions. 



The orientation that we use in this paper is less rigid: two edges labelled with 
the same variable can be oriented differently depending on the solutions involved 
(that is what we call non uniform orientation, see Figure [2 (b)[ ). Indeed we keep 

track of a set of 5 numbers associated with each variable and use it to discriminate among neighboring solutions. 
These 5 numbers provide information on the repartition of true and false occurrences of each variable in each type 
of clauses (clauses having 1, 2 or 3 true literals). Our intuition is that we should select solutions in which the least 
occurrences of true literals are critical. The less a clause has true literals, the more its true literals are critical. Such 
a property is by nature non uniform. 




(a) Uniform orientation. For exam- (b) Non uniform orientation, ob- 

ple b has 2 positive occurrences and tained in this example by minimiz- 

3 negative ones, so every edge la- ing 4/3i -I- 2/?2 + /Js (see definition in 

beled by b is oriented from 1 to 0. section [2]|. Both edges labeled by b 



are oriented differently (i.e. from 
to 1 as well as from 1 to 0). 



Fig. 2. Two different orientations for the solutions of formula Minimal solutions are in gray. 



We develop our technique in a general framework allowing us to apply it to a wide variety of 3-CNF models 
of formulas defined by their distributions; thus we derive new bounds for some known models of formulas [5]. The 
existence of other non uniform orientations that may give a smaller number of minimal elements and then better 
bounds remains to be investigated. 



In section [5] we present our framework and four different models of formulas; in section [3] we show how we make 
our non uniform selection of solutions, and sum up the bounds we obtain for each model. We give details on the 
calculation of the first moment and its constraints in section |4l as well as some hints on what led us to the weights 
we took for our non uniform selection. 



2 Definitions and notations 

We consider a generic random model of 3-CNF formulas having n variables and cn clauses. Models are parametrized 
by a probability distribution {dp^q)^ such that ^^j^ dp^q — 3c. In each model a satisfiability threshold will 
appear for a specific value of c we want to estimate. Before we get formulas we draw configurations as follows: 

1. each of the n variables is given p labelled positive occurrences and q labelled negative occurrences in a way that 
the overall proportion of variables with p positive occurrences and q negatives occurrences is dp q] 

2. a configuration can be seen as a matrix of Sen bins containing literals occurrences; the repartition of literals 
into the Sen bins is drawn uniformly among all (3cn)! permutations of labelled literals occurrences. 

A legal formula is a configuration where occurrences are unlabelled and each clause contains at most one occurrence 
of each variable. For the models we consider in this paper and described in section [2.11 it was shown that an upper 
bound on the satisfiability threshold obtained for configurations also applies to legal formulas (see [llj for the 
standard model and [2^ for models where p and q are bounded). So we shall work on configurations all along this 
paper. 

2.1 Overview of Models 

Standard Model: all literals are drawn uniformly and independently; it was shown in |12lllj that the resulting 



distribution is the 2D Poisson distribution: dr, „ = i^^'' 



'?,c\P+1 



v.g ^ K p I {p+qy. \ 2 1 

By analogy with the standard model we now define several other models where we force an equilibrium between 
variables occurrences and/or signs. These can be seen as regular variants of 3-SAT (just like regular graphs). The 
equilibrium cannot be perfect because of parity or truncation reasons, but we circumvent it as follows. Of course 
one can check that all of these distributions sum up to 1 and have an average of 3c. 



Model with Almost Balanced Signs: every variable appear with (almost) the same number of positive and 

(2p)! "p+l,p — "p,p+l — 2 (2p+l)! 



negative occurrences; we define dp^q by dp^p — - — tjtt^ — ^^"^ '^p+i,p — '^p,p+i = \^~T^+T)\ — (^^nd zero else- 



where) . 

Model with Almost Balanced Occurrences: every variable appear with (almost) the same number occur- 

rences; let t* = [3cJ and r* = 3c - t*] we define dp^q by dp^f-p = (1 - r*) and dp^t'+i-p = r* \t*+i 
(and zero elsewhere). 

Model with Almost Balanced Signs and Occurrences: every variable appear with (almost) the same num- 
ber occurrences and have strictly the same number of positive as negative occurrences (this model was examined 
in [2]); let p* = [^J and r* = ^—p*. We define dp, g by dp«_p» = 1 — r* and dp»+i,p*+i — r* (and zero elsewhere) . 

2.2 Types of clauses and variables 

Our selection method is based on different types of clauses: given any assignment, we call clause of type t a clause 
having t true literals under this assignment, and /3j the proportion of clauses of type t. 

Moreover we want to have some control on the number of occurrences of variables in the different types of 
clauses; to do so we need 6 numbers per variable, so we say that a variable is of type {i, j, k,l,m, v) if it is assigned 
V and has: 

false 



i true occurrences in clauses of type 1; 
j true occurrences in clauses of type 2; 

k true occurrences in clauses of type 3; ' 
1 false occurrences in clauses of type 1; 
m false occurrences in clauses of type 2; 
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/ 


I / 




/ 

m 


^ k 



Remark 1. For each variable we have i + j + k = p and I + m = q or vice versa (according to the value v assigned 
to the variable). 

Then we put some weights onto the solutions as follows: in a given solution each variable of type {i, j,k,l,m,v) 
receives a weight i-iJij^k,i,m,v The weight of a solution will be the product of the weights of all variables. It turns 
out that in the end we shall take binary weights, yielding in fact an orientation between solutions. We explain the 
choice of the weights in sections [3l and Then we apply the first moment method to the random variable X equal 
to the sum of the weights of the solutions. 



3 Selection of Solutions 

Let us recall how the first moment method works: we want to show that Pr {Y > a) is small but we don't have access 
to Pr {Y > a). Instead we use some EX. It suffices then to ensure that Pr {Y > a) < EX. For our problem 3-SAT, Y 
is the number of solutions, a = 1 and X is the total weight on the solutions. Since X > 0, Markov's inequality yields 
that Pr{X >l) <EX; so if we choose X such that Y > 1 imphes X > I, we have Pr (F > 1) < Pr (X > 1) < EX. 

Then our goal will be to tune the weights so that EX — > for the least ratio c = — — ^-rj — . 

^variaDies 



3.1 Construction of a Correct Weighting Scheme 

Of course we must put some constraints onto the weights in order that the weighting scheme can be correct for the 
first moment method: namely the sum of the weights of the solutions of a satisfiable formula must be at least 1. 
However the constraints we choose here might not be necessary for the first moment method to hold. 

Let us recall that given a solution, a variable is called free when the assignment obtained by inverting its 
value (0/1) remains a solution. Thus in our framework, a variable is free iff its i number is 0. How does the tuple 
(0, j,k,l,m, v) for a free variable x behave when the value v is inverted to l — v7i (x) <— 0, j (x) -f-i- I {x),k (x) o m (x) 
and V (x) ^ 1 — u (x). 

1. the first constraint we put is that uJij,k.i,m,v = 1 as soon as i > 1; that is, we put significant weights only onto 
free variables. The reason for this is that free variables allow to move between solutions. 

2. the second constraint is that 

1 ; (1) 

that is, the sum of the weights of a free variable in a couple of solutions differing only on that variable is 1. We 
impose this condition by analogy with the conditions on weights given in |13| . 

As suggested by the analysis given in section 14.41 we shall take (-^o,j,k,i,m,v — '^p{j.k.i,m.v) for a certain predicate 
P {j, k, I, m, v) linked with the sign of aipj^i + a^pk^m (where ai and are any real constants and p is an operator 
defined as pa^t = a — b). 

The fact that we imposed uJo,j.k.i,m.v + ^o,i,m,j,k,i-v = 1 tells us that given a solution and a free variable x at 
the value v, the predicate P is satisfied by x at the value v or (exclusively) by x at the value 1 — v. Thus we are 
able to define an orientation between neighboring solutions. 

Let us say that variable x is obedient when P is satisfied. We put an arc between 2 solutions differing only on 
1 (free) variable x from the solution Sd (where x is disobedient) to the solution So (where x is obedient), and we 
call that relation Sd > So- The notation > is not randomly chosen. 

Namely our weighting scheme counts 1 for a solution when it does not have any disobedient free variables, and 
otherwise; but what can ensure that whenever there is a solution, there is also a solution where all free variables 
are obedient? It suffices that the relation > is circuit-free. Then the transitive closure of > is an order, and we are 
precisely counting the minimal solutions in that order. Minimal solutions exist because the set of all solutions is 
finite. So let us see how we can make the relation > circuit-free. 



Recapitulation of Existing Methods. 

All Solutions: This method consists in computing the first moment on all solutions: P {j, fc, I, m, v) = 1. 

Negatively Prime Solutions (NFS): This method consists in counting only solutions which free variables are 
assigned 1. That is P {j, fc, l,m,v) = v > 0. This method was introduced in [8]. 

NFS with Imbalance: This method was introduced in and combined to some other ingredients in pTJ. This 
method consists in allowing free variables to take only a value such that the number of true occurrences is larger 
than the number of negative occurrences of this variable (and in case of equality, ties are broken in favor of 
the value 1). In other words P (j, k, I, m, v) = (pj^i + pk,m, >icx (0, 0), where >iox denotes the lexicographical 
order. 



Our Method. May we choose arbitrary real coefRcients ai and in the expression of aipj^i + azPk,m hi order 
that the first moment method should hold? It turns out that it is the case, and here is a proof of it. 

We make the following observation: how does the population of the 3 different types of clauses evolve when a 
free variable x is flipped? = pjj [x), 132+ = {pk,m - Pj,i) (x) and /33+ = -pk.m (x). 

Thus aipj.i + a^pk.m is the variation of aiPi — as/^s; so we may define our predicate P in the following way: 
P (j, k, I, m, v) = {aipj^i + Q!3Pfe,m, v) >icx (0, 0); thanks to v we break ties when aipj^i + azpk,m = 0, so that the 
underlying relation > between solutions is circuit-free: namely going from Sd to So when Sd > So strictly increases 
{-aiPi +azl33„v) for >icx- 

Moreover the exclusion between P (j, fc, Z, to, v) and P (l,m, j, k,l ~ v) is satisfied, which means that whenever there 
is a solution with a disobedient free variable, it suffices to flip the value of this variable so that it becomes obedient. 
We investigated the best ratio between ai and by numerical experiments. 

3.2 Summary of Results 



Table 1. Summary of our results. 
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our a 


a = 2.00 


1.01 < Q < 

1.16 


2.01 < a < 
2.24 


a > 1.01 



As one can see in table [TJ our method yields in all models a slight improvement on the bounds obtained by 
former methods. Note that for some models there is a range of values for a which give the same upper bound. 

In the model where signs as well as occurrences are balanced, the method of NPS+imbalance is of course the 
same as the method of NPS, whereas our method is somewhat better than the method of NPS. 

The bound we obtain in the standard model is 4.500; this is not better than the bound of 4.490 obtained by 
Diaz et ai in [TT] . Their calculation adds 2 ingredients to the method of jT2] : typicality of clauses and elimination 
of pure literals. These 2 ingredients might be combined to our approach to improve on the 4.490, but this would 
involve too complicated calculations with respect to the expected improvement. However in models where signs are 
balanced it is irrelevant to eliminate pure literals. 

4 The First Moment Method 
4.1 Types of variables 

We split the set of variables into several sets and subsets of variables. In order to be able to match the original 
random 3-CNF model of formulas where all literals are drawn independently, we should consider p and q to range 
in N. For convenience of our forthcoming maximization, we only take into account bounded values of p and q. So 
we are going to consider 2 kinds of variables, according to their numbers of occurrences. We follow the notations 
of [llj. We denote by M some integer whose value will be determined according to the required accuracy of the 
calculations; in practice we shall take M — 21. AI enables us to define 2 kinds of variables: 

1. the set of light variables, that is variables which indices are in the set 



jC = e N^^p <M Aq<M A dp_, > 0} 



(2) 



they are the most important variables since almost all variables are light in the models we consider; we call 
6p^q the proportion of light variables having p positive occurrences, q negative occurrences, and assigned 1. 
As a further refinement, we call Trij^k,i,m,v the proportion of variables of type (i, k,l,m,v) whose corre- 
sponding weight ujij^k,i,m,v is non zero, and omit the other ones because we shall need all active nij^k,i,m,v to 
be non zero. To connect TTij^k,i,m,v^s with 5p,q's we introduce the following set of tuples of integers: Qp,q = 
k, I, m) £ N^, i+j + k= pAl + m = g}; thus we have 

T^i,j,kJ,mS = Sp^q ; (3) 

{i,j,k,l,m}GQp,q 
(i,j,/c,Z,m)GQ,_p 

Note that equality U] involves Qq^p whereas equality [3] involves Qp^q. 
2. the set of heavy variables, that is all other variables; their indices are thus in the set 

7^ = {(p,q)eAr2,p>A/vq>MVdp,5 = 0} ; (5) 

we weaken the notion of satisfiability by considering that heavy variables are always satisfied, regardless of their 
signs and values. Doing so is harmless for the validity of the first moment method because we can only increase 
the number of solutions. In other words we are going to consider heavy variables as undistinguishable members 
of a tote bag. We call r the global scaled number of heavy variables: t = g)e-H '^p,q- 

We also need to distinguish some types of occurrences of heavy variables. We call H the global scaled number of 
occurrences of heavy variables: 

{p,q)eH (P + l) dp,q - 3c - J2(p,q)ec (P + l) d-p,q- Accordmg to the types of clauses where occurrences appear, 
H is divided into i?t's, where Ht is the scaled number of occurrences of heavy variables in clauses of type t. 
We are now ready to write down the expression of the first moment of X , the weight of all solutions. 



4.2 Expression of the First Moment and its Constraints 

We recall that all occurrences of literals are drawn according to the distribution dp^q (see section[5]). Thus the sample 
space we consider consists in the (3cn)! permutations of labelled occurrences of literals, and our parameters are 
n, c, dt^p, T, H and ujij^k,i,m,v''s (although we must carefully choose the weights LOi_j^k,i.m,v, as explained below in 
section l4.4p . 

All other quantities: Ht, St^p and Trij^k,i,m,v are variables, and the first moment of X can be split up into a 
big sum over all variables of the product of the following factors depending on variables: number of assignments, 
weight of an assignment and probability for an assignment to be a solution. 

1. number of assignments: each variable is assigned or 1: 2"^" Y[(p {s^'''n) ' 

2. weight of an assignment: Uip,q)ecIlit.j,k,Lm)eQ,,, '^I^j.fc'/X""; 

3. probability for an assignment to be a solution: quotient of the number of satisfied configurations by the total 
number of configurations: 

(a) number of satisfied configurations: a configuration can be seen as a set of bins filled with occurrences of 
literals: 

i. each of the 3cn bins is first given a truth value: 

there are ^acn)^'"^^^^^^'^" possibilities, and the following constraint appears: 

/3i + /32 + /33 = 1 • (6) 

ii. each light literal is given a tuple (i, j, fc, I, to) consistently with dp q and Sp^q. This gives a series of 
constraints: 



'^i.j,k,l.m,l ~^ ^ ^ '^i,j,k.l,m.O — ^p^q • (7) 



E 



Note that 6p^q = J2{i,j,k,i,m)eQp , '^i,j,k,i,m,i- Thus, given a family {TTij^k,i,m,v), there are 



n 

possible allocations. Moreover the following constraints appear, so that all occurrences of literals can fit 
into the destined types of clauses: 

iT^i,i,k,l,m,v + Hi = Pic ; (8) 

{p,q)ec 

{i,j,k,l,m)GQp,q 

ve{o,i} 

X] j'^i,j,k,l,m,v + H2 = 2p2C ; (9) 
(i,j,fe,i,m)eQp,<, 

ve{o,i} 

X] kTTij,k,i,m,v + Hz = 3/33C ; (10) 



(p,q)ec 

{i,j,k,l,m)£Qp^q 
"£{0,1} 



l'^i,i,k,l,m,v = 2Ac ; (11) 



(p.q)ec 

{i,j,k,l,m)eQp,q 
vG{0,l} 



rmrij^k,i,m,v = 1^20 . (12) 



{p.q)ec 

(i,j,k,l,m)eQp^q 

ve{o,i} 



iii. all occurrences of light variables are allocated to the 5 regions: 

n {p,q)ec (CZtk) Ctm))^''''"''''"'''" allocations are possible; 

{i,j,k,l,m)&Qp,g 
ve{0,l} 

iv. all occurrences of heavy variables are allocated to the 3 satisfied regions, which yields [fj^^ H^n) 
possible allocations; and we must add the following constraint: 

Hi+H2 + H3 = H . (13) 

V. all permutations of occurrences of literals are possible inside the 5 regions: 
their number is (/3icn)! (2/32cn)! (3/33cn)! (2/3icn)! (/32cn)!; 

(b) total number of configurations: the occurrences of literals can be in any order: (3cn)! permutations are 
possible. 

We denote by V the set of all families C of non negative numbers 



,{Hi,H2,H3),{Pi,(32,P3)\ (14) 

(i,j,k,l,m)eQp^g 

ve{o,i} 

satisfying the above constraints; note that V is convex (by linearity of constraints). We denote by I(n) the inter- 
section of V with the multiples of i; we get the following expression of the first moment: EX = X^^gx(n) l*^) 
where 



yHiti, H2n, H^^n J \(3icn, f32cn, P^^cn 
{j3icn)\ {2p2cn)\ (S/Jgcn)! (2picn)\ (/32cn)! 



(p,9)G£ 

n 



(3cn)! 

d„nn\ -i-r / ^, 



n n 



11 ^^'...M.™.. ,^ ■ (15) 



(p,g)G£ 
{i,j,k,l,m)eQp.q 
De{0,l} 



We get rid of all factorials thanks to the following Stiriing's inequalities due to Batir (^) y 27r (fc + < 

fc!<(i)V2.(fc+(i-l)). 

The boundedness of the set C of light variables (and thus the boundedness of the sets Qp,q) allows to write that 
T (n) < poly I (n) F"^ where 



2c 



n <r.' n ^^.M.^^^ ^r '^"M • (16) 

(i,j,fc,/,m)eQp,g 

ue{o,i} 

Once again, by the lightness property, X (n) consists of a bounded number of variables, each of which can take 
at most n + 1 values (as a multiple of i ranging between and 1). It follows that the size of X(n) is bounded by a 
polynomial polya (n). And since I (n) C V, we have EX < poly2 (n) poly^ (n) (max^g-p F)". 

4.3 Maximization of In F 

This is the technical part of our work. We mainly use the same techniques as 

1. In order to maximize ln_F under our constraints, we use the standard Lagrange multipliers technique. This is 
appendix 1^1 The following equations come from the Lagrange derivations and are important for our study: 

_ fi + j + k\fl + m\ 2i 3 I 2m /I Q^ 

V hJ.k J \l,m J 

xi, X2,yi and 2/2 are Lagrange multipliers, that is positive numbers; moreover rp,q is defined as follows: 

rp.q - ^ ; (19) 



A 



{iJ,k,l,m)£Qp^g 



2. In order to justify the use of this technique we must show that the function InF does not maximize on the 
boundary of the polytope of constraints; to do so we show that starting at a boundary point there is always a 
"good" direction inside the polytope which makes Ini^ greater. This is appendix [Bl 

3. Finally we must ensure that the solution we found by the Lagrange multiplier technique is indeed a global 
maximum; to do so we make a sweep over different values of the parameters Pt] indeed when these /3t are 
fixed the function In.F is strictly concave relative to the remaining variables, thus easier to maximize. This is 
appendix [C] 



4.4 Minimization of Global Weight 

Let us see how one can minimize F (or equivalently In.F) by a good choice of the weights. The following reasoning 
is not rigorous; we only aim at giving some hints to explain the choice of the weights we made in section [3l 

Remember that F is given by equation ll6l We want to minimize Ini^ by tuning the weights ujo.j,k,i,m,v, so we are 
going to differentiate Ini^ with respect to an individual woj',fe,i,m,i- Of course due to the constraints every variable 
depend on uJo,j,k,i,m,i in the process of maximizing hiF under these constraints. But we consider that the variations 
on all variables are negligible except for 7ro.j,fc,i.m,i (because of equation I17p and TTo^i^m,j.k,o (because of equations 
[T51 and [T|) ■ so we can write: 

d{\nF) d{\nF) dTro,j^k,i.m,i _^ d{\nF) dT:o.i^m.j,k,o 

,m,l 

Using equations [T71 [T51 and [Tl we find that: 



(21) 



a(lnF) 



■j + k\ 




- m 






ra 


'j + k\ 




- m 


.J.k ) 




ra 



rj+k,i+mxiy[yl"^ In (rj+k,i+r>ixiy[yl 



Now due to equations [THl and [201 and numerical experiments we make the following approximations: 
rj+k,i+mX2yiy'2^ ^ 1 and r jj^k,i+mX2yiv'2' ^ 1- the function x i— xln [ax) is strictly decreasing between and 

we can infer the following property: q^q^-'"^'' ^ > iff X2y{y2^ < x^y^.j/l'"' i-^- (f^) (^/a)*^ ™ < 1. 

Now let us consider we are at the minimum point of Ini^. If g^f "ff|^ ^ 7^ 0, then 

■^o,i,/c,;,m,i must be at the 

boundary, i.e. or 1 
a 

duio 



^^'"''^'^ ^ > iff aipj^i + aspk^m < 0, where ai = In ^ and = In (yl)- Thus: 



1. if aipjj + aspk^m < 0, then u}oj,k,Lm,i = 0; 

2. if aipjj + a3Pk,m > 0, then wo,i,fc,;,ma = 1; 

3. if OLiPj I + a^pk.m — 0, nothing can be said about woj,fc,/,)n,i- 



What about iOQ j k.i,m.o'^ 

1. if aipj^i + a3Pk,m < 0, then aipjj + a^p^n.k > 0, thus (^o,i,m,j,k,i = 1, so uJo,j,k,i,Tn,o 

2. if aipj^i + a3Pk,m > 0, then by the same argument, a;o,j,fe,z,m,o = 1; 

3. if aipj^i + aspk^m = 0, nothing can be said about iOoj^k,i,m,o- 



5 Conclusion 

We hope that the new track we opened will help gain some more insight and some more decimals in the quest of the 
3-SAT threshold. In particular note that we required the relation > between solutions to be circuit-free although 
this might not be necessary; indeed we only used the fact that this relation had at least one minimal element. 
The same remark holds for the constraints we put onto the weights of two neighboring solutions as introduced in 
equation [TJ since this might be too strong. Thus there may be better orientations or weighting schemes than ours. 
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Appendices 



Let us recall that we want to maximize the function: 





(p,g)e£ 
ve{0,l} 



n 




(23) 



on variables {TTij^k,i,m,v) {p,q)ec : {Hi, H2, H^) , /32, /Ja) subject to the following constraints: 

"£{0,1} 

/3i + /52 + /33 = 1 (24) 

^2 T^i,j,k,l,m,l + ^ ''^i,j,kj,m,0 = dp.q (25) 

{i,j,k,l,m)GQp,g {i,j,k,l,m)GQg^p 

Hi+H2+Hs= iP + <l)dp.i (26) 

i'^i,j,k,i,m,v + Hi= f^ic (27) 

(p,g)e£ 

De{o,i} 

X! j'^i,j,k,l,7n,v +H2= 2/32C (28) 

■"£{0,1} 

kTTtJ,k,l.Tn,v + i?3 = 3/33C (29) 



(p,9)e£ 

{i,j,k,l,m)eQp.q 

ve{os} 



lTri,j,k,l,m,v = 2^lC (30) 



(p,9)G£ 
(i,j,fc,i,m)eQp,, 
t)G{0,l} 

m'Ki^j^k,Lm,v = he (31) 

(p,<})G£ 
(i,j,fe,i,m)GQp,g 
■"£{0,1} 

To perform such a maximization we use the standard technique of Lagrange multipliers. 

A Resolution of the Global Lagrange Multipliers Problem 
A.l Elimination of redundant constraints 

The first thing to do is to remove redundant constraints. It appears that e.g. constraint is redundant with 
constraints (|27|) . (|29|) . (|30p. ((3T|) . because summing these 5 equations and using the previous ones (p4)) . ([26)), ((25]l 
gives a tautology: 

^ (i + j + fc + ; + m) ^^^^J,k,Lm,v +Hi + H2 + H3^3{f3i+f32 + P^) c 
(p,q)ec 

(i,j,k,l,m)£Qp,q 
f£{0,l} 

y^ ((i + j + fc) T^i^j.k.Lm.l + (l + m) TTi^j^k.Lm.l + {i + j + k) TTij^kJ.Tn.O + {I + m) TTi^-j^k.Lm.Q) + H = 3c 

{i,j,k,l,7n}eQp,q 



y^ ((« + j + fc) TTi j,fc,i,m,l + (/ + to) TTi,j,k,l,m,l) 



(i,j,fc,i,m)GQp,, 



+ ^ ((« + j + fc) TTij^fcJ^TO^O + (' + TO)7rij_fc_;^„,0) + ^ = 3c 



(p,9)e£ 

(i,j,k,l,m)^Qq^p 



y^ {pT^i,j,k,l,m,l + qT^i,j,k,l,m,l) + ^ {qT^i,j,k,l,m,0 + PT^i,j,k,l,m,o) + H — 3c 



(i,j,fc,i,m)GQp,g {i,3,k,l,m)^Qq^p 



(p,q)£C \{i,j,k,l,m)eQp^q {i,j,k,l,m)eQg,p 

{p,q)ec 

which was a requirement we made on (dp ^). 

Thus we get rid of constraint ()29p and there remain 7 constraints. 



H 



A. 2 Definition of the Lagrangian 

A = T\n2 + H \n H - Hi\n (^] - Hiln i^) - H^ln ] - H 



dp^q In dp 



/ p(i+j+k\(l+rn\\ 

El ( \ i,j,k ) \ Lm ) \ _ 



(i,j,k,l,m}GQp_q 
ve{Q,l} 



'^ij,k,l^m,v 



-2c In 3 + 2c/3i In ^2^^ + 2c/32 In (^2^^ + 2c/33 In (^3^^ + 2c 
+ (2chr6) (/3i+/32+/33-l) 

+ ^ (InT'p.q) I ^ T^i,j,k,l,m,l + ^ T^i,j,k,l,m,0 — dp^q 

{p,q}eC \{i,j,k,l,m)eQp_g {i,j,k,l,m)eQ^_p 

+ {\nh)lHi + H2 + Hi- iP + <l)dp,q 

( \ I 



(21na;i) 



E 



i,j,k,l,m,v + Hi — PiC 



{p,q)&C 
(i,j,k,l,m)^Qp^q 
\ f6{0,l} 



+ (lnyi) 



2/3ic 



{p,q)ec 

{i,j,k,l,m)£Qp^q 
\ -"£{0,1} 



(In X2) 



jT^i,j,k,l,m,v + H2 ~ 2/32C 



. (i,j,k,l,m)eQp,q 

\ fe{o,i} 



(21nj;2) 



Y ^'^^,3,k,l /32C 



(p,g)G£ 
{i,j,k,l,m)£Qp^q 



J 



A. 3 Derivatives with Respect to ni,j,k,i,m,v 

OA ffi+j + k\fl + m 

OT^i,j,k,l,m,l W JXhlTl 

+2i In + J In X2 + I hi j/i + 2m hi 1/2 

dA , / /i+ j + k\ fl + m 
= lnwi.j_fc_;.„,0 + In . ., , 

OT^i,j,k,l,m,Q W JKhm 

+2i In xi + j In a;2 + Z In yi + 2m In 2/2 



^"i^T^i,j,k,l,m,v + Inri+j+fc^i+m 

InTTij-^fc^j^m.t) + In ?';+m,i+j+fc 



i + J + fc\ fl + m 



fi+j + k\fl + m\ 2i 7 I 2m 



The Tp^q contraints become: 



{i,j,k,l,m)eQg,p 

Let us introduce 



Ef I + J + k\ fl + m\ 2i 1 I 2m 

Wjj,fe,;,m,il . ( \ri+j+k,i+mXi xi,yiy2 

(i,],k,l,m)eQp,g 

fi + j + k\ fl + m\ 2i 1 I 2m , 

+ ^i,3,k,l,mfiy ■ J ^ Jn+m,i+j+kXi X'2yiy2 = dp,q 



{i,j,k,l,m)eQp^g {i,j,k,l,m)EQg^p 

We have: 



'P,9 - 4 



Thus 



fi+j + k\fl+ m \ di+j+k,l+m 2i ] I 2m 
T^i,j,k,l,m,l = l^i,j,k,l,m,l\ . . , , } -. XiX2yi!j2 



i, i, k J \ l,m J Ai+j+k,i+r. 

fi+j + k\fl + m\ di+m,i+j+k 2i j I 2m 
T^i,3,k,l,mfi = <^i,j,k,l,m.fi I . . , I , "j Xi a;22/i2/2 

^,3,k J \ l,m J Ai+rn,i+j+k 



A. 4 Derivatives with Respect to /3t 

dA 

Wi 
dA 

W2 

OA 



= 2cln2 + 2cln/3i + 2cln6 - 2clna;i - 2clnwi 

^ ^ = 2c In 2 + 2c In /32 + 2c In 6 - 2c In a;2 - 2c In y2 

0P2 



2cln3 + 2cln/53 + 2cln6 



A - '''' 



2b 

X2y2 

2b 



The b constraint becomes: 



36 



, ^ xiyi X2y2 1 
2 2 3 



A. 5 Derivatives with Respect to Ht 

OA 



OA 

dA 



■IniJi +ln/i + 21nxi 
Tn if 2 + In + In X2 
AnHi + Xnh 



The h constraint becomes then: 



Hi = hxl 

H-2 = hX2 

Hs = h 



h{xl+X2 + l) = H 



Thus: 



H 



x{+X2 + l 



Hi 



Hxl 



Xi+X2 + 1 



Ho = 



HX2 



H, 



xf+X2+l 

H 

xl+X2 + l 



A. 6 Further Simplifications 

— 1st Moment: 



H 



H 



^ ^ fjHi rrff2 rrHs 11 '^p'.'q IT 
Hi H^ H^ 



ip,q)ec 
ve{o,i} 



/i+j+k\ /l+m\ 
\ i,j,k ) \ l,m ) 

'^i,j,k,l,m,v 



{2(3if' (2/32)^^ (3/3; 



n 



A 



l-\-m.i-\-j-\-k 



{p,q)eC ip,g)eC \di+j+k,l+mXiX2yiy2^ J (p,g)eC \dl+m,i+j+kXiX2yiy2™ 

{i,j,k,l,m.)&Qp,g {i,j,k,l,m)eQq,p 

2c 



H 



H 



— (xiyiY ' {x2y2) 



n 



(.i,j,k,l,m)eQq,p V"P.9^1 ^22/l2/2 



2c 



F = 2^ 



H 



H 



n 4 



^{i,j,k,l,m)eQp,g ^^•^•''•'•'^•^+^{i,j,k,l,m)eQg,p ^iJ.kJ.m.o^ 

dr,. 



„ V ^e{o,i} / V "£{0,1} > 

v&{o,i} ve{o,i} 

Vl i/2 
1 N 2c 

^^Xj X2 (/2 



\ TT / / -Ti-rt.n \ \ -^OR-,r- -.OR^r- -^OR^r- —OR^r- I ^ 



^-^Mt n U 



(p,9)e£ 



F = 2^ (a;^ + X2 
— Remaining constraints: 



rfp., tl2:l ^-2/3ic -2/32C -2^iic -2,32C _L , A „/32 , A 



(p,g)G£ ^'''^ (ij,fc,i,m)6Q5,p 



:2 +2:2 + 1 (£fl + £fl + i 



E 



^ A T. 

ip,q)eC ^''^ (i,3,k,Um)eQp,q 



(p,q)eC (ij,fc,i,m)eQ,,p 



E 



Pi*? 
A7; 



E 



m 



(p,9)G£ (jj,fc,/,m)GQp., \ ' / \ ' / 



- E ^ E u 

(p,q)e£ ^'"^ (ij,fe,/,m)eQ<,,p ^ ^ 



V 



E ' 



9A 



(p— G£^^''"''' 



i/a;2 



cf + 2:2 + 1 



E 



dA 



( xiVl I 3:21/2 I l\ 
V 2 ^ 2 ^3/ 

X2y2C 



dp,q V2 dAp,q _ X2y2C 

ip^ec ^^'^ 2 dy2 -2(^ + ^ + 1) 



Then we introduce Z = Y[{p q)ec ^fe' ''^^'^ Y = InZ: 



dY 2Hxi 

dXl x\ + .T2 + 1 

dY , H 
dx 



yic 

V 2 2 

y2c 



:2 xi +X2 + 1 + ^ + 



XiC 

Wi " (^ + ^ 
i9y 2:2 c 

dy2 



dy2 (^ + ^ + 1) 

To solve these equations we used Mathematical'^. The bound we obtained for c are summed up in table 



Table 2. Summary of results obtained by Lagrange's method. 



model 


standard 


balanced signs 


balanced 
occurrences 


balanced signs and 
occurrences 


our method 


4.500 


3.509 


4.623 


3.546 


(apj.i + Pk.m,v) > 










(0,0) 










our a 


2.00 


1.01 


2.01 


1.01 


Xl 


1.0083 


1.47787 


1.01694 


1.57726 


X2 


2.06625 


3.09005 


2.08256 


3.38506 


yi 


2.18256 


3.27457 


2.19038 


3.51076 




1.01253 


1.02742 


1.01221 


1.045 


/3i 


0.44373 


0.557479 


0.445306 


0.568436 


132 


0.421847 


0.365723 


0.421418 


0.363128 


P3 


0.134422 


0.0767974 


0.133276 


0.0684362 



B Inspection of the Boundary of V 

The boundary of V is reached when one of the variables is at 0. We want to be sure that F cannot be maximized 
by such a configuration. Remember that 

Ini^ = rln2 + i/lni7-i/iln^^^ -iJsln^^^^ - H-i\n{^^ -H 

+ ^ C?p,qlndp,g+ ^ T^i.j,k,l.m,v'^YL\^i.j,k,l.m,v — — j + 1 

t>G{0,l} 

-2cln3 + 2c/3iln ^2— ^ +2c/32ln^2— ^ +2c^3ln^3— ^ + 2c 

If we increase an Ht or a TTij,k.i,m.v from to a small ^ > and we change any other non zero variables, then 
the variation of InF is / = — ^ In^ + O (i^) is such that |- = — In^ + (1) — > +oo, so InF must increase; but what 
if we increase a /St from to a ^ > 0? Then |- = +ln^ + 6* (1) — > — oo. Thus the problem at the boundary of V 
comes from the f3f The technique will be, as in |12lll| . to make a small move in a well chosen direction in order 
to circumvent the negative side-effect of increasing a which is at 0. Such a direction will be referred to as an 
increasing direction. However we must ensure that such a direction is indeed in the polytope V. Note that in case 
we find the direction by pointing towards another point in V, this property results from the convexity of V. 

We used Mathematical'^ to minimize and maximize /3i under the above constraints in each model and our 
corresponding weighting scheme ; the precise bounds we obtained for /3i in each model are summed up in table ([3]) . 
Noteworthy is the fact that /3i can be neither nor 1 (thus we can have neither f3i = nor f32 = Ps = 0) . 



Table 3. Summary of bounds on 



model 


standard 


balanced signs 


balanced occurrences 


balanced signs and 
occurrences 


our method 


4.500 


3.509 


4.623 


3.546 


{apj^i + pk.m, v) > 
(0,0) 










our a 


2.00 


1.01 


2.01 


1.01 


our bounds for /?i 


0.177 < /3i < 0.912 


0.428 < /3i < 0.786 


0.182 < /3i < 0.909 


0.5 < I3i< 0.75 


(our bounds for /32 - 


< ^2 < 0.823 


< /?2 < 0.572 


< /?2 < 0.818 


< /32 < 0.5 


deductible from 










above) 










our bounds for /?3 


< ^3 < 0.412 


< /?3 < 0.286 


< /?3 < 0.409 


< /?3 < 0.25 



1. case where /32 — 0: then H2 = and TTij^k,i,m,v = unless j = m = 0; we call these variables forced as did 
moreover in the models where there are no heavy variables we consider variables Ht to be forced to as well. 

— subcase where there is an unforced variable at zero: we find a feasible point where /32 = and all unforced 
variables are non zero. Then a move towards this point gives an increasing direction (because /3i > and 
^3 > 0). To find such a point, we use the Lagrange multipliers method, as follows: 
• Definition of the lagrangian 

/l = rln2 + iJlniJ-iJiln^— ^ -i/gln^— ^ -H 

( i'tk) \ 

(i,0,fc,/,0)GQp., 

tie{o,i} 



-2cln3 + 2c/3i In + 2c/33 In + 2c 

+ (2c In 6) (/3i+/?3"l) 



(lnrp,q) ^ '^i,j,k,l,m,l+ ^ '^i-J,k,l 

(p,q)&C \(i,0,fc,/,0)GQp,5 (i,0,fc,/,0)eQ,,p 

(In/i) {H1+H3-H) 

( \ I 



(21na;i) 



(p,<j)e£ 
(i,o,fe,;,o)eQp,, 



(Inyi) 



lT^i,0,k,lfi,v — 2/3: 



ic 



(p,9)e£ 

(i,0,fe,/,0)6Qp,<, 



Derivatives with respect to 7ri.o,fc,!,o,« 



5yl 



lnw,;,o,fc,/,o,i + In 



Inwi o,fc,/,o,o + In 



i + k 
i, k 

i + k 
i, k 



In TTifl^kj.a.v + In r.i+k.i + 2i In + ^ In yi 



In ni^o,k,i,o,v + In n^^+fc + 2i In Xi + ^ In yi 



fi + k\ 2i I 

7ri.o,fc,i.o,i = '^i.o,fc,i.o,i . , Ki+fc.ia^i ?/i 

+ 2» I 

7ri,o,fc,i,o,o — '^i,o,fc,i,o,o I ^ ^ l^i.i+fea^i ?/i 



The Tp^q contraints become: 



(i,0,fc,g,0)eQp,<, 



i + k 
i, k 



i + k 



i^i,o,fc,p,o,o ( ^ ^ ]ri,i+kxf^yi ^ dp^q 
(i,0,fe,p,0)eQ,,p ^ ' ^ 



Let us introduce Ap^g = E(^,o,fc,9,o)eQp,, '^^o,fe,?,o,iG^Ja;f y? + ^ ; o^^g^ ^ Wi.o^fc^p.o.oG'J y^^^ 



^p,g^p,g — dp^q 



'p,q - A 

^p,q 



Thus 



''?: + k\ di+k.i 2i I 



i,k J Ai^k,i 
i + k\ d, 



i,k J Ai^ 



\,i+k 



• Derivatives with respect to Pt 

dA 



—— = 2cln2 + 2cln/3i + 2cln6 - 2c\nxi - 2clnwi 
=2cln3 + 2cln/33 + 2cln6 



The b constraint becomes: 



/3i 



xiyi 
2b 



^3 = 



36 



• Derivatives with respect to Ht 



xiyi , J_ ^ 1 
26 36 



6=^ + 1 
2 3 



dA 

dA 



■IniJi +ln/i + 21nxi 
■\nHs+\nh 



The h constraint becomes then: 



Hx = hxl 
Hs = h 



h{xl + l)=H 



• Remaining constraints: 



H 



^ ^ i[.^j]xfylaJifl,k,Qfi,i 

{p,q)eC P''' (i,0,fc,g,0)eQp,, ^ 



(p,g)e£ (i,0,fe,p,0)GQ,,p 



q \ 2i q , Hx{ XiViC 



, c'p.g ( l \ 2i p xiyic 

(p,9)e£ (i,0,fe,p,0)6Q„„ ^ '^ I— + 3J 



Then we introduce Z = ^pfg' and Y = hiZ: 



dY 2Hxi _ yic 



dY xic 

3) 



dyi + 



With Mathematical'^ we found the following solutions (and the corresponding values of InF): 



Table 4. Interior point when P2 = 0. 



model 


standard 


balanced signs 


balanced occurrences 


balanced signs and occurrences 


c 


4.500 


3.509 


4.623 


3.546 


Xl 


0.997334 


0.96803 


0.999726 


0.970462 


2/1 


2.07123 


2.06284 


2.05335 


2.06087 


InF 


-2.463 


-1.78313 


-2.53084 


-1.79349 



So all unforced variables are non zero. 

— subcase where all unforced variables are non zero: we define a function /(^) representing the variation of 

Ini^ under a small positive variation ^ in the following direction; remember that L^i.j,k.i.m.v = 1 as soon as 
i > 1 thus the corresponding variable T^i^j^k,i,m,v exists. Let us take some p > 3 and q>2 such that dp^q > 0. 
We make the following move: 

/3i ^ /3i - - 
c 

132^/32 + — 

c 
c 

7rp-l,0,l,g,0,l — > TTp-l, 0,1,9,0,1 " 5^ 
7rp-2,l,l,9,0,l 7rp-2,l,l,q,0,l + ^ 
TTp-l, 0,1,9-2,2,1 7rp-l,0,l,9-2,2,l + C 
7rp-l,l,0,9,0,l 7''p-l,l,0,9,0,l + 3^ 

so that all constraints remain satisfied; in fact wo arc performing a small move inside the polytope V 
and we would like to show that along this direction Ini*" is increasing. Note that /32 = 7i'p-2,i,i,g,o,i = 
TTp-i, 0,1,9-2, 2,1 = T^p-i,i,o,q,o,i = and all other variables here are non zero, so we have: 



c / \e 

+2c (^(33 - In (^^ (^/33 - ^)) - 2c/33ln (^^^3 
" (■"■p-i, 0,1, 9,0,1 — 5^) In (7rp_i 0,1,9,0,1 — 5^) + TTp^i 0,1,9,0,1 In 7rp_i 0,1,9,0,1 

- (7''p-2, 1,1, 9,0,1 + C) In (7rp_2,l,l,9,0,l +£.) - (7rp-l,0,l,9-2,2,l + C) In (7rp_i,o,l,9-2,2,l + 

- (7i"p-i,i,o,9,o,i + 3C) In (TTp.i, 1,0,9,0,1 +3^ + (0 



/(e) = +2c /3 



+4^ In 



+2c /3: 



In 



In 



/3i 



- 2c/3i In ( -/3i 



2c/33 In 



and thus: 



~ (7''p-i,o,i,q,o,i ^ 5^) In (7rp_i^o, 1,9,0,1 ~ 5^) + 7rp_i_o,i,g,o,i ^^'^p-i,o,i,q,o,i 
-2^ In ^-3^ In (30 + 0(0 



lim = lim (-InO + ©(1) 



Since limj_,.o ^-j^ = +oo, we have found an increasing direction. 
2. case where = 0: then Hs = and 'Kij^k,i.m,v = unless fc = 0; again we call these variables forced. 

— subcase where there is an unforced variable at zero: we find a feasible point where /Js = and all unforced 
variables are non zero. Then a move towards this point gives an increasing direction (because /3i > and 
/32 > 0). To find such a point we use again the Lagrange multipliers method, as follows: 
• Definition of the lagrangian 



El 
e 



H2ln\ 



ip,q)eC (p,q)eC 

ii,j,0,l,m)eQp,q 

-2cln3 + 2c/3iln ( 2— ) + 2c/32 In ( 2 



/ (i+3\ (l+m\ ■ 

E, / )\l,m) 
T^i,'jfi,l,m,v o O,/,m,t)- 

',q)ec \ 



+ 1 



2c 



■(2c In 6) (/3i+/32-l) 



{p.q)ec 
(In/i) {H1+H2-H) 
I 



7I'i,j,o,i,m,l + X 7'"i,j,0,;,m,0 — cip,9 

(i,j,0,l,m)eQp,q (i,i,0,i,m)eQ<,,p 



+ (21na;i) 



\ 



™i,3fi,l,m,v + Hi - PiC 



{i,j,0,l,m)eQp,q 

\ ve{o,i} 



( 



(In 2/1) 



(p,9)e/: 

ii,jfi,l,m)eQp,q 
\ "£{0,1} 



+ (Inaja) 
/ 



ip,q)eC 
{i,j,0,l,m)eQp^ 
\ "£{0,1} 



(2 In 2/2) 



y~l mTTijfl^l^rn,v " /32C 



(p,«)££ 
(i,j,0,;,m)eQp,, 
\ "£{0,1} 



• Derivatives with respect to 7ri,j,o,;,m,D 
dA 



dA 



lnwi,j,o,i,m,i +ln 
+2i In a;i + j In a;^ 
lnwi,j,o,i,m,o + In 



i + j\ fl + m 



^,3 



I, m 



+2i In xi + j In a;2 + 1 In yi + 2m In 7/2 
+ A /I + m 



ln7rij,o,i,m,w + lnri+TO,i+j 



+2i In a;i + jlnx2 + I In + 2m In j/2 



i + j\ fl + "l\ 2i j I 2m 

i+ j\ fl + m\ 2i j I 2m 

7ri,j,0,(,m,0 = i^i,j,0,l,m,0\ j j{ I ^ ]n+m,i+jXi a;2?/i1/2 



The rp^q contraints become: 



Efi+ j\ fl + m\ 2i j I 2m 



{i,j,0,l,m)eQp,g 

+ X! ^^i,j,o,i,m,o C^j^ i^l^m^ ri+m,i+jxfxiy{y2'^ = dp^q 



{i,j,0,l,m)eQg,p 



Let us introduce 



(i,,W,m)eQp., V«.J/V."^/ ii,j,o,i,m)eQg., \t,JJV,mj 

We have: 



* v. a — 



P,q - A 



Thus 



fi + j\ fl + m\ di+j^l+m 2i j I 2m 



i,j J \ l,m J Ai+j^i+r. 
' + j\fl + m\ dl+rn,%+, 
i,j J\l,m J Ai+rn,i+j' 



fi+ j\fl + m\dl+m,i+3 2i j I 2m 



• Derivatives with respect to /3t 

dA 



„ - 2cln2 + 2cln/3i +2cln6- 2clna;i - 2clnj/i 
^ = 2cln2 + 2cln/32 + 2cln6 - 2clna;2 - 2clnj/2 

OP2 



o X2y2 



The b constraint becomes: 



xiyi X2y2 ^ ^ 
2b 2b 



, xiyi , a;22/2 



• Derivatives with respect to Hf 



dA 

—— = -lni7i +lii/i + 21na;i 
= - \nH2 + lnh + \iiX2 

dH2 



Hi = hxl 

H2 = hX2 

The h constraint becomes then: 

h {xl +X2) = H 



H 

xl + X2 



• Remaining constraints: 



\^ dp,q -f <l \ f P \ 2t j I 2m , ^xf XiViC 



{p,q)eC P'"^ {i,j,0,l,m)eQg,p 

{p,q)eC P'" {i,j,0,l.ni)eQ„., ^ '-^z ^ ' / 

r^..^-«r- 0.^.0 'UJUW ^ ^'^'^ + + 0:2 - {xiyi+X2y2) 



(p,g)6£ " (i,i,0,i,m)6Q,,p 



{p,q)eC {i,j,0,l,m)eQg,p 

Then we introduce Z = n(p,g)e£ ^pf?" ^^^^ Y = hiZ: 



{xiyi + X2y2) 



dY 2Hxi yic 



dxi xi+X2 (^ + ^) 
dY ^ H _ y2C 



8X2 xl+X2 (^ + ^) 
dY xic 

^ = + 

With Mathematical'^ we found the following solutions (and the corresponding values of InF): 
So all unforced variables are non zero. 



Table 5. Interior point when /^a = 0. 



TTiori p1 








ha 1 a Tl PpH Q1 ffTl Q anH OPPll Vt'PTl PPQ 


c 


4.500 


3.509 


4.623 


3.546 


Xl 


0.512382 


0.473066 


0.493115 


0.494614 




0.546014 


0.489687 


0.501307 


0.494614 


yi 


0.583465 


0.520769 


0.531045 


0.494614 


V2 


0.529328 


0.513117 


0.505216 


0.494614 




-0.682149 


-0.375917 


-0.695819 


-0.33427 



subcase where all unforced variables are non zero: we define a function / (^) representing the variation of 
In^F under a small positive variation ^ in the following direction; remember that ujij^k,i,m,v = 1 as soon as 
i > I thus the corresponding variable TTij^k,i,m,v exists. Let us take some p > 3 and q > 2 such that dp^q > 0. 
We make the following move: 

/3i /?i + - 
c 

/32 -> /?2 - - 
C 

/33 ^ /33 + - 

c 

T^p-2,2,0,q-l,l,l 7I'p-2,2,0,g-l,l,l — 3^ 
0,1,9-1, 1,1 ^ ""p-l, 0,1, 9-1, 1,1 + 
7''p-2, 1,1,9,0,1 — > 7''p-2,l,l,(},0,l + 2^ 

SO that all constraints remain satisfied; in fact we are performing a small move inside the polytope V 
and we would like to show that along this direction Ini^ is increasing. Note that = 7rp_i,o,i,(j-i,i,i = 
i'p-2,1,1,9,0,1 = and all other variables here are non zero, so we have: 

/ (0 = 2c + In (^^ + ^) ) - 2c/3i In (^^A 

+2c (^/32 - ^) In (^^ (^/32 - ^) ) - 2C/32 In 0/32 

+2c (^/33 + ^) In (^^ (^A3 + ^ 

~ (■'l'p-2,2,0,9-1,1,1 ~ 3^) In ('7rp_2,2,0,9-l,l,l ^ 3^) + 7rp_2,2,0,9-l,l,l In 7rp_2,2,0,g-l,l,l 

- (TTp-i, 0,1,9-1, 1,1 + In (7rp-i,o,i,9-i,i,i + - (7i'p-2, 1,1, 9,0,1 + 2^) In (7rp_2, 1,1, 9,0,1 + 2^) + {£,) 

f (0 = +2c (^/3i + ^) In + f ) ) - 2c/3i In 

+2c (^/32 - ^) In (^^ (^/32 - ^) ) - 2C/32 In 0/32 
+2einfM 

~ (7rp-2,2,0,9-l,l,l — 3^) In (7rp_2,2,0,9-l,l,l ~ 3^) + 7rp_2,2,0,9-l,l,l In 7rp-2,2,0,9-l,l,l 

In e- 2^ In (20 + 0(0 



and thus: 



lim = lim (- ln£) + 6) (1) 



Since lim^_j.o = +00, we have found an increasing direction. 
3. case where all /3t > 0; suppose there is another variable at zero; we move towards the general solution we found 
in appendix|Xl where all variables are non zero. Then again lim^_>o = +00; so this is an increasing direction. 



C Inspection of the Interior of V 



As [TT] noticed in their calculation, we can perform a sweep over some coordinates in order to check that the 
solution of the Lagrange multipliers problem is indeed a global maximum. Namely when we fix all P2 (and 
(3^ — \ — Pi — (32), the function InF is strictly concave in the other variables. Let Vfj^^p^ the polytope where the 
remaining variables are allowed to move ; remember that the function to maximize is: 



\nF = T\n2 + H\nH - Hi\n \ — ) - H2 In ( — ) - i/g In ( — ) - 



el V e / V e 



, , , , I V i,j,k l\ l,m) 



i>e{0,l} 

-2cln3 + 2c/3i In (^2—^ + 2c/32 In (^2—^ + 2c^3 In (^3—^ + 2c 

If we increase an Ht or a 7Tij^k,i.m,v from to a small 1^ > and we change any other non zero variables, then 
the variation / of InF: / = — ^In^ + (^) is such that |- = — In^ + 69 (1) ^ +00, so InF must increase; thus the 
function cannot maximize on the boundary of Vjj^^p^ and we can apply the Lagrange multiplier technique again. 
But now by strict concavity of the objective function, we know that the solution we find corresponds to a global 
maximum. 

— Definition of the lagrangian 



A = T\n2 + H\nH - Hi\n[ - H2\n(^\ -iJglnf^ I -H 



^ ^ dp^q^^dp^q 4" ^ ^ ^ij^fc,/,??!,!! In I ^i.j ,k,l,7n,v 



\ i,j.k / V l.7n / 



(p,9)e£ 



{i,j,k,l,m)£Qp^, 



(^^i,j,k,l,m,v 



-2cln3 + 2c/3i In (^2^^ + 2c^2 In (^2^ j + 2c^3 In (^3^ | + 2c 

{p,q)eC \{i,j,k,l,m)eQp.q {i,j,k,l,m)eQq,p 



J'p,q 



(Inh) [ Hi + H2 + H3 - J2 iP + l)d: 

( \ 



+ (21na;i) 



'i'^i,j,k,l,m,v + Hi — PiC 



{p,q)ec 

. (i-j,k,l,m)GQp_g 
\ ve{QA} 



( 



+ (In 2/1) 



lT^i,j,k,l,m,v — 2/3lC 



(p,</)e£ 

{i,j,k,l,m)(iQpq 
\ "6{0,1} 



(In 2:2) 



(2 In 2/2) 



jT^i,j,k,l,m,v + H2 - 2/32C 



(p,g)e£ 
. {i,j.k,l,m)eQp.q 

\ fe{04} 



(p,</)e£ 

\ "£{0,1} 



Derivatives with respect to TTij,k,i,m,v 



As in the general case we find that 



'i + j + k\(l + m\ di+j+k,l+m 2t 3 I 2m 



Derivatives with respect to 

As in the general case we find that 



Remaining constraints: 



hj,^ J \ l,m J Ai+j+k,i+m 
fi+j + k\/l + m\ di+,„.,+j+k 2i j I 2m 



Hi 

H2 

Ha 



Hxl 



x\+X2 + l 

HX2 
x\ + X2 + l 

H 

xl+X2 + l 



2Hxi 


2/3ic 


x\+X2 + l 


Xi 


H 


_ 2p2C 


x\+X2 + l 


X2 


dY 




dyi 


yi 


dY 


2/?2C 


dy2 


2/2 



— Objective function: 



Hi H^ ^^^^^^^ ^ '^^M,m,v J \6 J 

ii,j,k,l,m)GQp,q 
ve{0,l} 




So we made a sweep over Pi and /32 in the feasible region and plotted the maximum point given as the solution 
of these equations, which confirmed the fact that the solutions to the global Lagrange system are indeed global 
maxima. 



Table 6. Summary of results obtained by Lagrange's method. 



model 


standard 


balanced signs 


balanced 
occurrences 


balanced signs and 
occurrences 


our method 

{apj^i + pk,m,v) > 

(0,0) 


4.500 


3.509 


4.623 


3.546 


01 


0.44373 


0.557479 


0.445306 


0.568436 


:h 


0.121817 


0.3()572:i 


0.121118 


0.3():!128 


/33 


0.134422 


0.0767974 


0.133276 


0.0684362 




Fig. 3. Maximum of InF for different values of /3i and /32 in the standard model at c = 4.500. Numerically we found 
that the maximum is at /3i ~ 0.44313 and ^2 ^ 0.421847. 




Fig. 4. Maximum of In F for different values of /3i and /32 in the model with balanced signs at c = 3.509. Numerically 
we found that the maximum is at /3i ~ 0.557479 and /32 — 0.365723. 
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Fig. 5. Maximum of InF for different values of /3i and [32 in the model with balanced occurrences at c = 4.623. 
Numerically we found that the maximum is at f3i ~ 0.445306 and /32 — 0.421418. 
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Fig. 6. Maximum of InF for different values of /3i in the model with balanced signs and occurrences at c = 3.546. 
In this particular model where each variable has as many positive occurrences as negative ones, true and false 
surfaces are equal: /3i + 2/^2 + 3/3^ = 2/3i + /32, thus f32 = 1-5 — 2/3i. Numerically we found that the maximum is at 
/3i ~ 0.568436 and /32 c± 0.363128. 



